Can one identify non-equilibrium in a three-state system by analyzing two-state 

trajectories? 
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For a three-state Markov system in a stationary state, we discuss whether, on the basis of data 
obtained from effective two-state (or on-off) trajectories, it is possible to discriminate between an 
equihbrium state and a non-equilibrium steady state. By calculating the full phase diagram we 
identify a large region where such data will be consistent only with non-equilibrium conditions. 
This regime is considerably larger than the region with oscillatory relaxation, which has previously 
been identified as a sufficient criterion for non-equilibrium. 
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For the emerging field of thermodynamics of small 
systems, or stochastic thermodynamics^"— , enzymes and 
proteins like molecular motors constitute paradigmatic 
systems since their conformational changes can be ob- 
served on a single molecule level using a variety of exper- 
imental techniques^. The analysis of trajectories record- 
ing the transition bctvi^cen microstates of the enzyme can 
reveal insights into both the underlying kinetics and the 
thermodynamic implications of such changes^"—. If the 
surrounding conditions like temperature or concentra- 
tions of other reactants are constant, the enzyme reaches 
a stationary state for which two classes must be distin- 
guished. First, in a genuine equilibrium state, no net 
flux between any two microstates of the enzyme occurs. 
Second, in a non-equilibrium steady state (NESS), the 
probability to find the enzyme in any microstate is still 
time- independent. However, non-zero net fiuxes between 
microstates indicate that the system is externally driven 
with concomitant entropy production. If all microstates 
are experimentally accessible, in principle, distinguish- 
ing between these two alternatives is trivial, given long 
enough trajectories. It suffices to infer both the station- 
ary probabilities p| for the system to be in state i and 
the rates kij for a transition between i and j, and then 
to check for genuine equilibrium in which the detailed 
balance condition p^kij = Pjkji for any two states must 
hold. 

In practice, rarely all relevant microstates are ex- 
perimentally accessible or distinguishable. The result- 
ing only coarse-grained information affects and aggra- 
vates a thermodynamic consistent analysis of individual 
trajectories^'^"*. In standard applications, often only a 
two-state trajectory is obtained in which each observ- 
able state may in fact contain several microstates. In 
an ambitious program, Flomenbom and co-workersi^"— 
have investigated which information about the underly- 
ing topology of the state space, or 'network', can be in- 
ferred from the observation of such two-state trajectories. 
But even if the underlying topology is known, two-state 
trajectories cannot reveal the full information like the 
value of all rates between microstates. 

For time-independent external conditions, arguably, 
the most relevant question is, whether or not such an 



enzyme acts in an equilibrium state or in a NESS. The 
simplest system for which this problem can be posed 
is a three-state system like the stochastic Michaelis- 
Menten scheme for an enzyme turning substrate into 
a produc1i^ '^°i^* . Suppose that two of the three states 
are experimentally not distinguishable, can we decide in 
which of the two classes of stationary states the enzyme 
is in, if we are given a sufficiently long two-state trajec- 
tory? It is well-known that oscillations in the correla- 
tion function of such a two-state trajectory imply that 
the underlying three-state system is in a NESS rather 
than in equilibrium^^. Here, we show that the range of 
ascertained non-equilibrium can be extended much be- 
yond this oscillatory regime, i.e., even for decaying cor- 
relations it can be possible to infer an underlying NESS 
state. However, there will remain parameters for which 
the question posed in the title cannot be answered af- 
firmatively. Only if all three states are visible, like in 
dual-color fiourescence correlation spectroscopy, one can 
discriminate between equilibrium and non-equilibrium 
steady states^^i^i. 

We consider a three-state Markov system with states 
i = 1,2,3 and transition rates kij. The dynamics of the 
probability pi {t) to be in state i at time t then is governed 
by the master equation 
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We assume that state 1 (the on-state) is fluorescently 
labelled and can be observed, but that it cannot be de- 
termined experimentally, whether the system is in state 
2 or state 3 (together called the Oj(f -state), as sketched in 
fig. [Tj The experiment then yields an effective two-state 
trajectory which no longer follows a Markovian dynam- 
ics. 

Four quantities can be extracted from such two-state 
trajectories and identified with their theoretical corre- 
spondences as follows: 
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Fig. 1: Three-state Markov model with states 2 and 3 lumped 
into a compound state. 

(i) The waiting time distribution in state 1 is exponen- 
tial with a decay rate 

L = (fcl2 + fci3). (3) 

(ii) The waiting time distribution lyoff(i) in the ojj- 
state can be calculated from a modified Markov model 
where state 1 is treated as an absorbing state which 
amounts to setting the rate constants ki2 and ki^ to zero. 
This yields the effective rate matrix 

\ k23 -fc31 - fc32 J ' ^ ' 

which governs the time evolution of the probabilities p2 (t) 
and P3{t) to be in state 2 and 3, respectively, if we stop 
the dynamics when the system reaches state 1. The wait- 
ing time distribution then is the negative time derivative 
of the compound probability to be either in state 2 or 
state 3, 

wMt) = ^j^iP2{t)+Mt))- (5) 

This waiting time distribution is a sum of exponentials 
with exponents given by the eigenvalues of the modi- 
fied rate matrix K which can be calculated as 

A, - ^ [-S ± Vs^ - r) (6) 

with 

S* = ^21 + + ^31 + ^32 and (7) 

T = 4(A:2lfc31 + fc2lfc32 + ^23^31). (8) 

If both decay times can be determined from experimental 
time traces, both quantities S and T can be inferred. 

(iii) Last, one can measure the time-dependent proba- 
bility pi{t) to be in state 1 with initial condition pi(0) — 
1. This quantity can also be calculated from the master 
equation ((T]), yielding a sum of exponentials 

=Pi + cie^i* + C2e^^*. (9) 

The exponents Xi are eigenvalues of the rate matrix K 
([2]) and can be written as 

A. = ^ (-L - 5 ± ^{S + LY-T- m) (10) 



with 

M= 4fci2(fc23 + fc3i-Hfc32) 
+4fci3(/c21 +fc23 + fc32)• 
The prefactors ci_2 can be obtained from the linear com- 
bination of eigenvectors of the matrix K which yield the 
initial state pi(0) — 1. 

In summary, the four quantities L, 5, T and M can be 
determined from experimental time traces, provided the 
statistics is good enough to allow fits parametrizing the 
given exponential behavior. In principle, one could de- 
rive some other set of four independent quantities rather 
than L, S*, T and M. Our choice is convenient due to 
two reasons. First, all four quantities are non-negative 
(even positive if no rate constant kij vanishes). Second, 
for the distinction between equilibrium and a NESS the 
absolute time-scale is irrelevant. This fact can be used to 
scale all times by 1/i leading to the three dimensionless 
quantities 

S = S/L, f = T/L^ and M = M/L^. (12) 

A rescaling of time in the original model would lead to 
five independent rate constants kij = kij/L (with ki2 + 
fci3 1). Hence given measured reduced rates S, T and 
M there still remains a two-parameter manifold in the 
three-state system dynamics that is consistent with the 
observed two-state trajectory. 

The three-state system is in equilibrium if and only if 
the original rates obey the condition 

^12^23^31 = ^32^21^13- (13) 

An analysis of the functional dependence of the rates 
kij on S, T , M and the equilibrium condition 
assisted by Wolfram's algebraic software package 
MATHEMATICA 7 leads to the identification of three 
regimes in the phase diagram shown in fig. [21 It is checked 
whether or not for given reduced rates S and M positive 
rate constants kij compatible with (jl3p can be found. 
This is the case only if 

<T s^Ti= MS - — , (14) 

which defines the regime denoted EQ/NESS in fig. [51 In 
this regime, the measured data is principally insufficient 
for discriminating a NESS from an equilibrium state. 

Hence for any measured T with T > Ti the system is 
definitely in a NESS. This criterion constitutes our main 
result. We now show that this parameter space is much 
larger than the set of parameters for which oscillations in 
the correlation function occur. The latter property has 
previously been identified as a sufficient criterion for a 
NESS. Oscillations in the autocorrelation function pi{t) 
^ occur if the decay rates Xi in eq. (fTOj) acquire an 
imaginary part, i.e., if 

f >f2 = (5 + 1)2 -M. (^5) 
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Fig. 2: (color online) (a)-(c): Phase diagram in the (M, T) plane for different values of S with (a) 5* — 0.1, (b) S = 1.5 and (c) 
S = 10. The white area EQ/NESS allows both equilibrium states and NESSs, the purple area NESS can contain only NESSs 
and the brown area OSC corresponds to NESSs with oscillatory correlation function. The OSC regime touches the EQ/NESS 

regime at points ^2 + 2S, — 1 + 5*^^ . The checkered area is excluded since no choice of non-negative rate constants will lead to 

such an (Af, T) value. The green line represents equilibrium states with some of the rate constants equal to zero. The black 
lines are the projection of the subdomain of the phase diagram shown in (d) that corresponds to the given 5* value, 
(d): Phase diagram in the (fc23, fcsi) plane for fixed rate constants fcis — 1, k2i = 1, ^32 = 1 and ki2 = 4. For each point in 
the diagram, the parameters L, S, T and M are calculated and the three regimes accordingly identified. The blue hyperbola 
comprises the genuine equilibrium state according to eq. (|13|) . 



This regime is denoted by OSC in fig. [51 It is easy 
to check that f2(M) ^ Ti(M) (with equality only for 
M = 2{S+ 1)). For all measured f with f 2 ^ f > fi, 
denoted by NESS in fig. [21 our new criterion establishes 
a NESS on the basis of two-state trajectories which do 
not exhibit oscillatory behaviour. While for 5^1 the 
newly established region of NESSs beyond the oscillatory 
regime is rather small, see fig. , both for 5 <g; 1 and 



for S* 3> 1 the present analysis shows a large regime of 
newly ascertained NESSs. In the latter two cases, the 
oscillatory regime is, for S ^ 0.5, totally absent or rather 
small, see figs. Ufa) and (c), respectively. 

For fixed S the phase diagrams of physically admissi- 
ble (M, f ) values are restricted to A? 45 and f ^S^, 
since values outside this range would require negative kij 
values. Moreover, for M ^ 2S, only T ^ Ti can be re- 
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alized with non- negative kij. The region ^ T > Ti, 
which is excluded by the latter condition, is shown check- 
ered in figs. [211 a)- (c). The equalities in these constraints 
of physically admissible values hold only if some of the 
rates are zero. ^ 

Finally, an analysis of the boundary Ti{M), which re- 
quires some rate constants kij to be zero, shows that for 
^ Af ^ 25* the system is definitely in equilibrium on 
this boundary. For T = the system is also definitely in 
equilibrium. 

For representative examples of these results in the orig- 
inal space of the six rates kij, consider fig. [5fd). There 
we show the three regimes EQ/NESS, NESS and OSC in 
the (^23, fcsi) plane holding the other four rate constants 
fixed. The line of genuine equilibrium ()13p is embedded 
into region EQ/NESS where information from the state 
trajectory is not sufficient to uniquely identify equilib- 
rium. However for (^23, k^i) parameter values in the 
region NESS our analysis predicts correctly a NESS be- 
yond the previously established OSC region. 

Experimental measurements of the waiting time distri- 
bution Woff(0 ® s-'^d autocorrelation function pi{t) ^ 
will cause some uncertainty due to finite data. For a 
rough estimate, we assume a relative error of ±e in all 
eigenvalues ^ and (fTO]) , while it should be possible 
to determine the decay time L ^ with better precision. 
A linear error propagation then yields a constant relative 
error for S and T of ±£ and ±2e respectively. The ef- 
fect on M is a relative error of ±(2e + 4£T/A/). Thus 
error bars of experimentally determined pairs of (A/, T) 



values would increase linearly on straight lines through 
the origin in fig. [2l[a)-(c). Since the maximum value of T 
increases quadratically with the maximum value of M , 
the uncertainty on the phase boundary between NESS 
and EQ/NESS in fig. [ll^a)-(c) increases with increasing 
S. Hence, a clear discrimination between the two regions 
would be affected by errors less in the small S regime than 
for large S. For the data shown in fig. [2l^a) and e — 0.05, 
the phase boundary effectively broadens to about 11% 
on the top and 10% at the bottom. 

In summary, we have derived a criterion sufficient to 
identify a non-equilibrium steady state in a three-state 
system based on information extracted from sufficently 
long two-state trajectories. While the new criterion ex- 
tends the region of ascertained NESSs significantly be- 
yond the previously identified region based on oscillation 
in the autocorrelation function, there remain parameter 
values for which two-state trajectories contain principally 
not enough information to discriminate NESS from an 
equilibrium state. It will be interesting to perform a 
similar analysis for four-state systems or even more com- 
plex networks, where one would expect that the region 
of ascertained NESSs on the basis of two-state trajec- 
tory information becomes relatively smaller than in the 
three-state case. Generalization of our approach to cases 
where three-state trajectories, obtained, e.g., via dual- 
colour fiuorescence, are available on four or more-state 
networks is feasible in principle, but may, in practice, 
become algorithmically challenging. 
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